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Abstract 

We present information-theoretic definitions and results for analyzing symmetric-key en¬ 
cryption schemes beyond the perfect secrecy regime, i.e. when perfect secrecy is not attained. 
We adopt two lines of analysis, one based on lossless source coding, and another akin to rate- 
distortion theory. We start by presenting a new information-theoretic metric for security, called 
e-symbol secrecy, and derive associated fundamental bounds. This metric provides a parame¬ 
terization of secrecy that spans other information-theoretic metrics for security, such as weak 
secrecy and perfect secrecy. We then introduce list-source codes (LSCs), which are a general 
framework for mapping a key length (entropy) to a list size that an eavesdropper has to resolve 
in order to recover a secret message. We provide explicit constructions of LSCs, and show that 
LSCs that achieve high symbol secrecy also achieve a favorable tradeoff between key length and 
uncertainty list size. We also demonstrate that, when the source is uniformly distributed, the 
highest level of symbol secrecy for a fixed key length can be achieved through a construction 
based on minimum-distance separable (MDS) codes. Using an analysis related to rate-distortion 
theory, we then show how symbol secrecy can be used to determine the probability that an eaves¬ 
dropper correctly reconstructs functions of the original plaintext. More specifically, we present 
lower bounds for the minimum-mean-squared-error of estimating a target function of the plain¬ 
text given that a certain set of functions of the plaintext is known to be hard (or easy) to infer, 
either by design of the security system or by restrictions imposed on the adversary. We illustrate 
how these bounds can be applied to characterize security properties of symmetric-key encryp¬ 
tion schemes, and, in particular, extend security claims based on symbol secrecy to a functional 
setting. Finally, we discuss the application of our methods in key distribution, storage and 
privacy. 
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1 Introduction 


The security properties of a communication scheme can, in general, be evaluated from two funda¬ 
mental perspectives: information theoretic and computational. For a noiseless setting, uncondi¬ 
tional (i.e. perfect) information-theoretic secrecy can only be attained when the communicating 
parties share a random key with entropy at least as large as the message itself [3]. Consequently, 
usual information-theoretic approaches focus on physically degraded models [3], where the goal is 
to maximize the secure communication rate given that the adversary has a noisier observation of 
the message than the legitimate receiver. On the other hand, computationally secure cryptosys¬ 
tems have thrived both from a theoretical and a practical perspective. Such systems are based on 
yet unproven hardness assumptions, but nevertheless have led to cryptographic schemes that are 
widely adopted (for an overview, see [5]). Currently, computationally secure encryption schemes 
are used millions of times per day, in applications that range from online banking transactions to 
digital rights management. 

Computationally secure cryptographic constructions do not necessarily provide an information- 
theoretic guarantee of security. For example, one-way permutations and public-key encryption can¬ 
not be deemed secure against an adversary with unlimited computational resources. This is not to 
say that such primitives are not secure in practice - real-world adversaries are indeed computation¬ 
ally bounded. There are, however, cryptographic schemes that are believed to be computationally 
secure and simultaneously provide some security guarantee against computationally unbounded 
adversaries, albeit such guarantee is not absolute secrecy. This was noted by Shannon [3] and later 
by Heilman [6] in a companion paper to his and Diffie’s work “New directions in Cryptography” 

[Zl- 

Our goal in this work is to characterize the fundamental information-theoretic security prop¬ 
erties of cryptographic schemes when perfect secrecy is not attained. We follow the footsteps of 
Shannon and Heilman and study symmetric-key encryption with small keys, i.e. when the length 
of the key is smaller than the length of the message. In this case, the best a computationally 
unrestricted adversary can do is to decrypt the ciphertext with all possible keys, resulting in a list 
of possible plaintext messages. The adversary’s uncertainty regarding the original message is then 
represented by a probability distribution over this list. This distribution, in turn, depends on both 
the distribution of the key and the distribution of the plaintext messages. 

We evaluate the information-theoretic security in this setting through two complementary lines 
of analysis: (i) one based on lossless source coding, where the security properties of the uncertainty 
list are measured using mutual information-based metrics and secure communication schemes are 
provided based on linear code constructions, and (ii) another akin to rate-distortion theory, where 
the mutual information-based metrics are translated into restrictions on the inference capabilities 
of the adversary through converse results. We describe each approach below. 
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1.1 Lossless Source Coding Approach 

If perfect secrecy is not achieved, then meaningful metrics are required to quantify the level of 
information-theoretic security provided by a cryptographic scheme. We define a new metric for 
characterizing security, e-symbol secrecy, which quantifies the uncertainty of specific source sym¬ 
bols given an encrypted source sequence. This metric subsumes traditional rate-based information- 
theoretic measures of secrecy which are generally asymptotic [1]. However, our definition is not 
asymptotic and, indeed, we provide a construction that achieves fundamental symbol secrecy 
bounds, based on maximum distance separable (MDS) codes, for finite-length sequences. We note 
that there has been a long exploration of the connection between coding and cryptography [8], and 
our work is inscribed in this school of thought. 

We also introduce a general source coding framework for analyzing the fundamental information- 
theoretic properties of symmetric-key encryption, called list-source codes (LSCs). LSCs compress a 
source sequence below its entropy rate and, consequently, a message encoded by an LSC is decoded 
to a list of possible source sequences instead of a unique source sequence. We demonstrate how any 
symmetric-key encryption scheme can be cast as an LSC, and prove that the best an adversary can 
do is to reduce the set of possible messages to an exponentially sized list with certain properties, 
where the size of the list depends on the length of the key and the distribution of the source. Since 
the list has a size exponential in the key length, it cannot be resolved in polynomial time in the key 
length, offering a certain level of computational security. We characterize the achievable e-symbol 
secrecy of LSC-based encryption schemes, and provide explicit constructions using algebraic coding. 

1.2 Rate-Distortion Approach 

While much of information-theoretic security has considered the hiding of the plaintext, crypto¬ 
graphic metrics of security seek to hide also functions thereof [9]. More specifically, cryptographic 
metrics characterize how well an adversary can (or cannot) infer functions of a hidden variable, and 
are stated in terms of lower bounds for average estimation error probability. This contrasts with 
standard information-theoretic metrics of security, which are concerned with the average number of 
bits that an adversary learns about the plaintext. Nevertheless, as shown here, restrictions on the 
average mutual information can be mapped to lower bounds on average estimation error probability 
through rate-distortion formulations. 

Using a rate-distortion based approach, we extend the definition of e-symbol secrecy in order to 
quantify not only the information that an adversary gains about individual symbols of the source 
sequence, but also the information gained about functions of the encrypted source sequence. We 
prove that ciphers with high symbol secrecy guarantee that certain functions of the plaintext are 
provably hidden regardless of computational assumptions. In particular, we show that certain 
one-bit function of the plaintext (i.e. predicates) cannot be reliably inferred by the adversary. 

We illustrate the application of our results both for hiding the source data and functions thereof. 
We provide an extension of the one-time pad [3] to a functional setting, demonstrating how certain 
classes of functions of the plaintext can be hidden using a short key. We also consider the privacy 
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against statistical inference setup studied in m, and show how the analysis introduced here sheds 
light on the fundamental privacy-utility tradeoff. 

From a practical standpoint, we investigate the problem of secure content caching and distribu¬ 
tion. We propose a hybrid encryption scheme based on list-source codes, where a large fraction of 
the message can be encoded and distributed using a key-independent list-source code. The infor¬ 
mation necessary to resolve the decoding list, which can be much smaller than the whole message, 
is then encrypted using a secure method. This scheme allows a significant amount of content to be 
distributed and cached before dealing with key generation, distribution and management issues. 

1.3 Related work 

Shannon’s seminal work [3] introduced the use of statistical and information-theoretic metrics 
for analyzing secrecy systems. Shannon characterized several properties of conditional entropy 
(equivocation) as a metric for security, and investigated the effect of the source distribution on the 
security of a symmetric-key cipher. Shannon also considered the properties of “random ciphers”, 
and showed that, for short keys and sufficiently long, non-uniformly distributed messages, the 
random cipher is (with high probability) breakable: only one message is very likely to have produced 
a given ciphertext. Shannon defined the length of the message required for a ciphertext to be 
uniquely produced by a given plaintext as the unicity distance. 

Heilman extended Shannon’s approach to cryptography [6] and proved that Shannon’s random 
cipher model is conservative: A randomly chosen cipher is likely to have small unicity distance, 
but does not preclude the existence of other ciphers with essentially infinite unicity distance (i.e. 
the plaintext cannot be uniquely determined from the ciphertext). Indeed, Heilman argued that 
carefully designed ciphers that match the statistics of the source can achieve high unicity distance. 
Ahlswede m also extended Shannon’s theory of secrecy systems to the case where the exact source 
statistics are unknown. 

The problem of quantifying not only an eavesdropper’s uncertainty of the entire message but 
of individual symbols of the message was studied by Lu in the context of additive-like instanta¬ 
neous block ciphers (ALIB) [12H14| . The results presented here are more general since we do not 
restrict ourselves to ALIB ciphers. More recently, the design of secrecy systems with distortion 
constraints on the adversary’s reconstruction was studied by Schieler and Cuff m- We adopt 
here an alternative approach, quantifying the information an adversary gains on average about the 
individual symbols of the message, and investigate which functions of the plaintext an adversary 
can reconstruct. Our results and definitions also hold for the finite-blocklength regime. 

Tools from algebraic coding have been widely used for constructing secrecy schemes [8]. In 
addition, the notion of providing security by exploiting the fact that the adversary has incomplete 
access to information (in our case, the key) is also central to several secure network coding schemes 
and wiretap models. Ozarow and Wyner m introduced the wiretap channel H, where an adversary 
can observe a set k of his choice out of n transmitted symbols, and proved that there exists a code 
that achieves perfect secrecy. A generalized version of this model was investigated by Cai and 
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Yeung in where they introduce the related problem of designing an information-theoretically 
secure linear network code when an adversary can observe a certain number of edges in the network. 
Their results were later extended in A practical approach was presented by Lima et al. in 

|22| . For a survey on the theory of secure network coding, we refer the reader to [23] , 

The list-source code framework introduced here is related to the wiretap channel II in that a 
fraction of the source symbols is hidden from a possible adversary. Oliveira et al. investigated in 
[23] a related setting in the context of data storage over untrusted networks that do not collude, 
introducing a solution based on Vandermonde matrices. The MDS coding scheme introduced in 
this paper is similar to [23|, albeit the framework developed here is more general. 

List decoding techniques for channel coding were first introduced by Elias [25| and Wozencraft 
[26j . with subsequent work by Shannon et al. [271128] and Forney [29|. Later, algorithmic results 
for list decoding of channel codes were discovered by Gurusuwami and Sudan m- We refer the 
reader to m for a survey of list decoding results. List decoding has been considered in the context 
of source coding in [32|. The approach is related to the one presented here, since we may view a 
secret key as side information, but [32] did not consider source coding and list decoding together 
for the purposes of security. 

The use of rate-distortion formulations in security and privacy settings was studied by Ya¬ 
mamoto [33] and Reed [M]. Information-theoretic approaches to privacy that take distortion into 
account were also considered in [T01I35II37] . 


1.4 Notation 

Throughout the paper capital letters (e.g. X and Y) are used to denote random variables, and 
calligraphic letters (e.g. X and y) denote sets. All the random variables in this paper have a 
discrete support set, and the support set of the random variables X and Y are denoted by X and 
y, respectively. For a positive integer j,k,n, j < k, [re] = {!,..., re}, [j, fc] = {}, j -|- l,...,fc}. 
Matrices are denoted in bold capital letters (e.g. H) and vectors in bold lower-case letters (e.g. 
h). A sequence of re random variables Vi,..., Y„ is denoted by Y”. Furthermore, for J C [re], 
X'^ = ..., where ik & J and ii <12 < ••• < i\j\. Equivalently, for a vector 

X = (xi,..., Xn), ..., Eor two vectors x, z € M”, we denote by x < z the 

set of inequalities {xi < Furthermore, we denote by In{t) the set of all subsets of [re] of size 

t, i.e. J E Init) 77 C [re] and \ J\= t. 

All the logarithms in the paper are in base 2. We denote the binary entropy function as 


hb{x) = —X log X — {1 — x) log(l — x). 

The inverse of the binary entropy function is the mapping : [0,1] ^ [0,1/2] where 




X, 


0 < X < 1/2 


1 — X, otherwise. 
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The set of all unit variance functions of a random variable X with distribution px (denoted by 
X px) is given by 


^ 2 {Px) = {</>: T" M such that \\cj){X)\\ 2 = 1, X ~ px} , 


where ||(/)(X)|| 2 = ^/^(p(X)^. 

The operators Tx and Ty denote conditional expectation and, in particular, {Tx o g){x) = 
E [g{Y)\X = x] and (Ty o f){y) = E [f{X)\Y = y], respectively. For two random variables X and 
Y, the minimum-mean-squared error (MMSE) of estimating X from an observation of Y is given 
by 


mmse(X|y) = 


mm 


E 


(X-Xf 


1.5 Communication and threat model 

A transmitter (Alice) wishes to transmit confidentially to a legitimate receiver (Bob) a sequence 
of length n produced by a discrete source X with alphabet A and probability distribution px- 
We assume that the communication channel shared by Alice and Bob is noiseless, but is observed 
by a passive, computationally unbounded eavesdropper (Eve). Both Alice and Bob have access 
to a shared secret key K drawn from a discrete alphabet JC, such that H{K) < H{X'^), and 
encryption/decryption functions Enc : X"' x JC ^ Ai and Dec : Ai x 1 C ^ where Ai is the set 
encrypted messages. Alice observes the source sequence X^, and transmits an encrypted message 
M = Enc(X"',A'). Bob then recovers X^ by decrypting the message using the key, producing 
X^ = Dec(M, AT). The communication is successful if X” = X”. We consider that the encryption 
is closed O pg. 665], so Dec(c, fei) / Dec(c, ^2) for ki^k2 G JC, ki ^ k2- We assume Eve knows the 
functions Enc and Dec, but does not know the secret key, K. Eve’s goal is to gain knowledge about 
the original source sequence. 


1.6 Organization of the paper 
1.6.1 Symbol secrecy 

We introduce the definitions of absolute and e-symbol secrecy in Section [2J Symbol secrecy quan¬ 
tifies the uncertainty that an eavesdropper has about individual symbols of the message. 


1.6.2 Encryption with key entropy smaller than the message entropy 

We present the definition of list-source codes (LSCs), together with fundamental bounds, in Section 
[3l Practical code constructions of LSCs are introduced in Section [H We then analyze the symbol 
secrecy properties of LSCs in Section [5l 


7 





1.6.3 A Rate-Distortion View of Symbol Secrecy 

In Section [6] we introduce results for characterizing the information leakage of a security system 
in terms of functions of the original source data. In particular, we derive converse bounds for 
the minimum-mean-squared error (MMSE) of estimating a target function of the plaintext given 
that certain functions of the plaintext are known to be hard (or easy) to infer. We illustrate the 
application of these bounds in a generalization of the one-time pad. We also use these results to 
bound the probability of error of estimating predicates of the plaintext given that a certain level of 
symbol secrecy is achieved. 

1.6.4 Further applications and practical considerations 

Section [3 presents further applications of our results to security and privacy, together with prac¬ 
tical considerations of the proposed secrecy framework. Finally, Section [5] presents our concluding 
remarks. 


2 Symbol Secrecy 


In this section we define e-symbol secrecy, an information-theoretic metric for quantifying the 
information leakage from security schemes that do not achieve perfect secrecy. Given a source 
sequence A”’ and a random variable Z dependent of A"’, e-symbol secrecy is the largest fraction 
tjn such that, given Z, at most e bits can be learned on average from any t-symbol subsequence of 
A”. We also prove an ancillary lemma that bounds the average mutual information between A” 
and Z in terms of symbol secrecy. 

Definition 1. Let A” be a random variable with support A"", and Z be the information that leaks 
from a security system (e.g. the ciphertext). Denoting = {Aj}jgy, we say that px^,z achieves 
an e-symbol secrecy of gLe{X'^\Z) if 


/Ue(A”|Z) = max 


\J\ 


< e vy c [n],0 < \ J\< t 


In particular, the absolute symbol secrecy of A” from Y is given by 


^o(A"|Z) = max < — 

' n 


/(A'^;Z) = 0 Vy C [n],0< |y|<t 
We also define the dual function of symbol-secrecy for and Z as: 


( 1 ) 

( 2 ) 


e*(A'‘|Z) ^ inf {e > 0 I Pe{X^\Z) > t/n} . 


(3) 


The next examples illustrate a few use cases of symbol secrecy. 

Example 1. Symbol secrecy encompasses other definitions of secrecy, such as weak secrecy [38], 
strong secrecy [39| and perfect secrecy. For example, given two sequences of random variables 





X'^ and Z"', if —)• 1 for all e > 0, then ^ —)• 0. The converse is not true, as 

demonstrated in Example [3] below. Furthermore, = 0 if and only if i 2 q{X^\Z^) = 1. 

Finally, the reader can verify that I{X'^] Z^) —?■ 0 if and only if there exists a sequence = o(n) 
such that /re„(X"'|Z"') ^ 1. 


Example 2. Consider the case where X = {0,1}, X"^ is uniformly drawn from T"', and Z is the 
result of sending X"^ through a discrete memoryless erasure channel with erasure probability a. 
Then, for any J [n], J ^ 0, 


\J\ 


(1 - a). 


and, consequently. 


^^e{X^\Z) 


0 , for 0 < e < 1 — a, 
1 , e > 1 — a. 


Example 3. Now assume again that X^ is a uniformly distributed sequence of n bits, but now 
Z = Xi. This corresponds to the case where one bit of the message is always sent in the clear, and 
all the other bits are hidden. Then, for any J C [n] such that {1} G ^7, 


I{X^-Z) = l, 


and, for 0 < e < 1, 




Consequently, a non-trivial symbol-secrecy cannot be achieved for e < 1. In general, if a symbol 
Xi is sent in the clear, then a non-trivial symbol secrecy cannot be achieved for e < H(Xi). Note 
that /(X”; Z)ln —>■ 0, so weak secrecy is achieved. 

Example 4. We now illustrate how symbol secrecy does not necessarily capture the information 
that leaks about functions of X"^. We address this issue in more detail in Section O Still assuming 
that X"^ is a uniformly distributed sequence of n bits, let Y be the parity bit of X"", i.e. Z = 
n”=i(—Then, for any J 0 [n], 

I{X^;Z)=0, 

and, for 0 < e < 1, 

^.{X^\Z) = 

n 

and, for e > 1, fif:{X^\Z) = 1. 

The following lemma provides an upper bound for Z) in terms of /ie(X”’|Z) when X*^ is 

the output of a discrete memoryless source. 

Lemma 1. Let X^ be the output of a discrete memoryless source X, and Z a noisy observation of 
X". For any e such that 0 < e < H{X), if IjL^{X^\Z) = u*, then 


-I(X”; Z) < H{X) - u*{H{X) - e). 
n 


( 4 ) 
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Proof. Let /ie(X”'|Z) = u* = t/n, J G X„(t) and J = [n]\J. Then 


Z) = Z) + -liX^] Z\X^) 

n n n 

< u*e+ ^^~*^ H{X) 
n 

= H{X)-u*{H{X)-e), 

where the first ineqnality follows from the definition of symbol secrecy, and the second ineqnal- 
ity follows from the assnmption that the source is discrete and memory less and, consequently, 
I{X^-Z\X^)<H{X^\X^) = {n-t)H{X). □ 

The previous result implies that when ^^{X^\Z) is large, only a small amount of information 
about X"^ can be gained from Z on average. However, even if I{X^-, Z) is large, as long as 
is non-zero, the uncertainty about X^ given Z will be spread throughout the individual symbols 
of the source sequence. This property is desirable for symmetric-key encryption and, as we shall 
show in Section El can be extended to determine which functions of X”' can or cannot be reliably 
inferred from Z. Furthermore, in Section [5] we introduce explicit constructions for symmetric- 
key encryption schemes that achieve a provable level of symbol secrecy using the list-source code 
framework introduced next. 

3 LSCs 

In this section we present the definition of LSCs and derive fundamental bounds. We also demon¬ 
strate how any symmetric-key encryption scheme can be mapped to a corresponding list-source 
code. 

3.1 Definition and Fundamental Limits 

We introduce the definition of list-source codes is given below. 

Definition 2. A (2"'^, n)-LSC {fn,gn,L) consists of an encoding function /„ : A"- e-)- [2""^] 

and a list-decoding function gn^L ■ [2"^] P(A’"')\0, where V{X^) is the power set of A"' and 

\gn,L{w)\= Vtc G [2”-^]. The value R is that rate of the LSC, L is the normalized list size, 

and is the list size. 

Note that 0 < L < 1. From an operational point of view, L is a parameter that determines the 
size of the decoded list. For example, L = 0 corresponds to traditional lossless compression, i.e., 
each source sequence is decoded to a unique sequence. Furthermore, L = 1 represents the trivial 
case when the decoded list corresponds to A’"'. 
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Figure 1; Rate list region for normalized list size L and code rate R. 

For a given LSC, an error is declared when a string generated by a source is not contained in 
the corresponding decoded list. The average error probability is given by 

e(/„,5n,L) = Pr(X" i gn,Lifn{Xn))- (5) 

Definition 3. For a given discrete memoryless source X, the rate list size pair {R,L) is said 
to be achievable if for every (5>0,0<e<l and sufficiently large n there exists a sequence 
of (2"'^", |T’|’^^",n)-list-source codes {(/n, such that < i? + (5, |L„ — L\< 6 and 
e{fn, 9n,Ln) < e- The rate list region is the closure of all rate list pairs {R,L). 

Definition 4. The rate list function R{L) is the infimum of all rates R such that (R, L) is in the 
rate list region for a given normalized list size 0 < L < 1. 

Theorem 1. For any discrete memoryless source X, the rate list function is given by 

R{L) = H{X)-Llog\X\ . (6) 

Proof. Let (5 > 0 be given and {{fn, gn,L„)}^=i be a sequence of codes with (normalized) list size 
L„ such that Ln ^ L and for any 0 < e < 1 and n sufficiently large 0 < e{fn,gn,Ln) < e- Then 

Pr(x"€ U gn,L„H]>RT{X^Ggn,LMn{Xn)) (7) 

\ ujeW" / 

> 1-6 ( 8 ) 

where = [2^^"] and Rn is the rate of the code {fn, gn,Ln)- There exists no(S,e, IT’D where if 
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n > no{5, e, |<^|), then 


Rn + Ln\og\X\ = - log 

n ' 


^log( Y1 bn,L„M|j 

VuiGW" / 


1 


> - log 
n 


U 9n,Lr.{w) 

> H{X) - 6, 


(9) 


where the last inequality follows from @01 Lemma 2.14]. Since this holds for any (5 > 0, it follows 
that B(L) > H{X) — Llogj^Lj for all n sufficiently large. 

We prove achievability next. Let 0 < L < 1 be given, and let = [uLj. Furthermore, let X"' 
be a sequence of n source symbols, and denote the first nL„ source symbols and 

the last n(l —L„) source symbols where we assume, without loss of generality, that nL is an integer. 
Then, from standard source coding results HD pg. 552], for any e > 0 and n sufficiently large, and 
denoting an — ]'nL„(iL(X)+ e)]/n, j3n — \n{l — Ln){H{X) + e)\/n, there are (surjective) encoding 
functions 

^ [2-“i and ^ [2^^ 

and corresponding (injective) decoding functions 


ffn,i : [2’"“"] ^ and : [2”^"] ^ 

such that Fv{gl,{f^^jX-^-)) / X^^-) < 0(e) and ^ x(i-^")-) < 

0(e). 

For w G [2"^’*] and x G T”, let the list-source coding and decoding functions be given by 
/n(x) ^ 

ffn,L„(^) = {x G T” : G [2’"“’*] such that (/^^(xl^^l),/2(i_^)(x[”'^+^’’"l)) = (u,u;)}, 


respectively. Then 

Pr (x- G 9n,lSfniXn)) > Pr (<i(/^l(^^")) = ^ 

>l-0(e). 


Observe that the rate-list pair achieved by {fn,9nL ) i^mLn) = (/dn, o,i/logjXj)). Conse¬ 
quently, 


Rn<il-Ln)iH{X) + e) + n-^ 
< iL(X) -|- e — Un 
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= H{X) + e-Ln\og\Xl 

where the second inequality follows from < Ln{H{X) + e) + n~^. Observe that ^ n(l — 
L)H{X) + e = R. Since L„ —)■ L{H{X) + e)/log|d:l|= L as n —)• oo, by choosing n sufficiently large 
the rate-list pair {R, L) can be achieved, where R and L satisfy 

R<H{X) + e-L\og\X\. 

Since e is arbitrary and L can span any value in [0,//(X)/log|d:l|], it follows that R{L) < H{X) — 
Llog|Al|. □ 

3.2 Symmetric-Key Ciphers as LSCs 

Let (Enc, Dec) be a symmetric-key cipher where, without loss of generality, Ai = [2"'^] and Enc : 

X K, ^ M. and Dec : M. x K, ^ X^. Then an LSC can be designed based on this cipher by 
choosing k' from /C and setting the encoding function fn{^) = Enc(x, k'), where x G X^, and 

9n,L{fn{^)) = {z G Tl” : 3/c G /C such that Enc(z, k) = /n(x)}, 

where L satisfies |/C|= \X\'^^. If the key is chosen uniformly from /C then the decoded list corre¬ 
sponds set of possible source sequences that could have generated the ciphertext. The adversary’s 
uncertainty will depend on the distribution of the source sequence X'^. 

Alternatively, symmetric-key ciphers can also be constructed based on an (2"^, | n)-list- 
source code. Let {fn,gn,L) be the corresponding encoding/decoding function of the LSC, and 
assume that the key is drawn uniformly from /C = [lAl”-^], where the normalized list size L deter¬ 
mines the length of the key. Without loss of generality, we also assume that Alice and Bob agree on 
an ordering of X and, consequently, X^ can be ordered using the corresponding dictionary ordering. 
We denote pos(x) the position of the source sequence x G A in the corresponding list 5n,L(/n(x)), 
where pos : X^ — )■ [lAl*^^]. 

The cipher can then be constructed by letting the message set be Ai' = [2^^^] X and, for 

X G A” and /c G /C, 

Enc(x, k) = (/n(x), (pos(x) -P k) mod |/C|). 

For (a, b) G Ai' , the decryption function is given by 

Dec((a, 6 ), fe) = {x : /n(x) = a, pos(x) = {b — k) mod |/C|}. 

In this case, an eavesdropper that does not know the key k cannot recover the function pos(x) and, 
consequently, her uncertainty will correspond to the list 5'n,L(/n(x)). 
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4 LSC design 


In this section we discuss how to construct LSCs that achieve the rate-list tradeoff ([ 6 ]) in the 
finite block length regime. As shown below, an LSC that achieves good rate-list tradeoff does 
not necessarily lead to good symmetric-key encryption schemes. This naturally motivates the 
constructions of LSCs that achieve high symbol secrecy. 

4.1 Necessity for code design 

Assume that the source X is uniformly distributed in Fg, i.e., Pr(A = x) = 1/g Vx G Fg. In this 
case R{L) = (1 — L) logf;. A trivial scheme for achieving the list-source boundary is the following. 
Consider a source sequence = (A^, X^), where X^ denotes the first p = n — [Ln\ symbols of X^ 
and A* denotes the last s = [Ln\ symbols. Encoding is done by discarding A®, and mapping the 
prefix A^ to a binary codeword of length nR = \n— [Ln\ logg] bits. This encoding procedure 
is similar to the achievability scheme used in the proof of Theorem [TJ 

For decoding, the codeword is mapped to A^, and the scheme outputs a list of size 
composed by A^ concatenated with all possible combinations of suffixes of length s. Clearly, for n 
sufficiently large, i? « (1 — L) logg, and we achieve the optimal list-source size tradeoff. 

The previous scheme is inadequate for security purposes. An adversary that observes the 
codeword Y"^^ can uniquely identify the first p symbols of the source message, and the uncertainty 
is concentrated over the last s symbols. Assuming that all source symbols are of equal importance, 
we should spread the uncertainty over all symbols of the message. Given the encoding /(A”), a 
sensible security scheme would provide /(Aj; /(A”)) < e log (7 for 1 < i < n. We can naturally 
extend this notion for groups of symbols or functions over input symbols, which is what symbol 
secrecy captures. 

4.2 A construction based on linear codes 


Let A be an i.i.d. source with support A and entropy H{X), and {sn,rn) a source code for A with 
encoder Sn ■ —>■ F™" and decoder : F™" —)• A”. Furthermore, let C be a {mn,kn,d) linear 

cod^ over Fg with an {nin — kn) x irin parity check matrix H„ (i.e. c G C H„c = 0). Consider 
the following scheme, where we assume 


kn = nL„ log|A|/logg 


is an integer, 0 < < 1 and Lr, 


L as n —>■ 00 . 


Scheme 1. Encoding: Let x„ G A” be an n-symbol sequence generated by the source. Compute 
the syndrome cr„ through the matrix multiplication 


— Hn'Sn(x^ra) 


^For an overview of linear codes an related terminology, we refer the reader to Ba¬ 
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and map each syndrome to a distinct sequence of nR = \{mn — kn) log o'] bits, denoted by ynR- 
Decoding: Map the binary codeword YnR to the corresponding syndrome cr^. Output the list 

9n,L„{(^n) = {rn(z)|z S F™", Cr„ = H„z} . 

Theorem 2. If a sequence of source codes {(sn, is asymptotically optimal for source X, 

i.e. mn/n^ H{X)/\ogq with vanishing error probability, schemeU\ achieves the rate list function 
R{L) for source X. 

Proof. Since the cardinality of each coset corresponding to a syndrome is exactly 

|ffn,L„(cr„)|= q^", 


the normalized list size is 


Ln = log|;t’| = (fcnlogg)/(nlog|T|). 

By assumption, Ln ^ L as n ^ oo. Denoting mn/n = H[X)/\ogq + 6n, where 6n ^ 0 since the 
source code is assumed to be asymptotically optimal, it follows that the rate of the LSC is 

Rn = \imn - kn)logq]/n 

= \{H{X) + 6nlogq)n - Lnulogl^l^/n 
H{X) - L\og\X\, 

which is arbitrarily close to the rate in ([UD for sufficiently large n. □ 

The source coding scheme used in the proof of Theorem [2] can be any asymptotically optimal 
scheme. Note that if the source X is uniformly distributed in Fg, then L„ = kn/n and any message 
in the coset indexed by (t„ is equally likely. Hence, Rn = (n — k)\ogq/n = H{X) — Llogq, which 
matches the upper bound in ([6]). Scheme [T] provides a constructive way of hiding information, and 
we can take advantage of the properties of the underlying linear code to make precise assertions 
regarding the security of the scheme. 

With the syndrome in hand, how can we recover the rest of the message? One possible approach 
is to find a /c„ x n matrix D„ that has full rank such that the rows of D„ and H„ form a basis of 
F™". Such a matrix can be easily found, for example, using the Gram-Schmidt process with the 
rows of H„ as a starting point. Then, for a source sequence x„, we simply calculate t„ = D„x„ 
and forward to the receiver through a secure channel. The receiver can then invert the system 



and recover the original sequence x„. This property allows list-source codes to be deployed in 
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practice using well known linear code constructions, such as Reed-Solomon [42 1 Chap. 5] or Random 
Linear Network Codes [l3l Chap. 2]. 

Remark 1. This approach is valid for general linear spaces, and holds for any pair of full rank 
matrices H„ and D„ with dimensions (m„ — x and kn x m^, respectively, such that 
rank([H^ = rrin. However, here we adopt the nomenclature of linear codes since we make 

use of known code constructions to construct LSCs with provable symbol secrecy properties in the 
next section. 

Remark 2. The LSC described in scheme [T] can be combined with other encryption methods, 
providing, for example, an additional layer of security in probabilistic encryption schemes ([SllH]). 
A more detailed discussion of practical applications is presented in Section [71 

5 Symbol Secrecy of LSCs 

We next present fundamental bounds for the amount of symbol secrecy achievable by any LSC 
considering a discrete memoryless source. Since any encryption scheme can be cast as an LSC, 
these results quantify the amount of symbol secrecy achievable by any symmetric-key encryption 
scheme that encrypts a discrete memoryless source. 

Lemma 2. Let {{fm gn)}^=i be a sequence of list-souree codes that aehieves a rate-list pair {R, L) 
and an e-symbol secrecy of /ig He as n —)• oo. Then 0 < < min | 

Proof. We denote Note that, for J' C [n] and \ J\= 

= nia,^nH{X)- H{X-^\Y^^-) 

where the last inequality follows from the definition of symbol secrecy and < |y|e = 

npLe^n^. Therefore 

y^,,n{H{X) - e) < -H{X^\Y^^-) 
n 

< Lniog|y|. 


The result follows by taking n ^ oo. □ 

The previous result bounds the amount of information an adversary gains about particular 
source symbols by observing a list-source encoded message. In particular, for e = 0, we find a 
meaningful bound on what is the largest fraction of input symbols that is perfectly hidden. 

The next theorem relates the rate-list function with e-symbol secrecy through the upper bound 
in Theorem [2j 


16 



Theorem 3. If a sequence of list-source codes {{fn, 9n,Ln)}^=i achieves a point {R', L) with 
—)• = Ce for some e, where R' = lim„,_ 5 .oo then R' = R{L). 

Proof. Assume that {{fn, gn,L„)}'^=i satisfies the conditions in the theorem and (5 > 0 is given. 
Then for n sufficiently large, we have from 

= -I(X'^-,Y^^) 
n n 

< H{X) - c,iH{X) - e) + 5 
= HiX) - Llog\X\+6. 

Since this holds for any <5, then R' < H{X) — LloglTl. However, from Theorem [H R' > H{X) - 
L log IT I, and the result follows. □ 

5.1 A scheme based on MDS codes 

We now prove that for a uniform i.i.d. source X in Fg, using scheme [1] with an MDS parity check 
matrix H achieves Since the source is uniform and i.i.d., no source coding is used. 

Proposition 1. If m is the parity check matrix of an {n,k,d) MDS code and the source X" is 
uniform and i.i.d., then Scheme\^ achieves the upper hound no = L, where L = k/n. 

Proof. Let C be the set of codewords of an {n,k,n — k + 1) MDS code over Fg with parity matrix 
H, and let x € C. Fix a set H € Dn(k} of k positions of x, denoted x'^. Since the minimum 
distance of C is n — k + 1, for any other codeword in z G C we have ^ x^. Denoting by 
C^ = { gf^^ : X G C}, then |C'^|= |C|= (^. Therefore, contains all possible combinations of 
k symbols. Since this property also holds for any coset of H, the result follows. 

□ 


6 A Rate-Distort ion View of Symbol Secrecy 

Symbol secrecy provides a fine-grained metric for quantifying the amount of information that leaks 
from a security system. However, standard cryptographic definitions of security are concerned not 
only with what an eavesdropper learns about individual symbols of the plaintext, but also which 
functions of the plaintext an adversary can reliably infer. In order to derive analogous information- 
theoretic metrics for security, in this section we take a step back from the symmetric-key encryption 
setup and study the general estimation problem of inferring properties of a hidden variable X from 
an observation Y. More specifically, we derive lower bounds for the error of estimating functions of 
X from an observation of Y. By using standard converse results (e.g. Fano’s inequality [m Chap. 
2]), symbol secrecy guarantees are then translated to guarantees on how well certain functions of 
the plaintext can or cannot be estimated. 

We first derive converse bounds for the minimum-mean-squared-error (MMSE) of estimating a 
function cj) of the hidden variable X given Y. We assume that the MMSE of estimating a set of 
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functions ^ given Y is known, as well as the correlation between 4>j{X) and (j){X). 

Bonnds for the MMSE of (t){X) are then expressed in terms of the MMSE of each 4>j{X) and the 
correlation between 4>{X) and 4>j{X). We also apply this result to the setting where 4> and ijij are 
binary fnnctions, and present bounds for the probability of correctly guessing 4>{X) given Y. These 
results are of independent interest, and are particularly useful in the security setting considered 
here. 

The set of functions d> can be used to model known properties of a security system. Eor example, 
when X is a plaintext and T is a ciphertext, the fnnctions 4>j may represent certain predicates of 
X that are known to be hard to infer given Y . In privacy systems, X may be a user’s data and Y a 
distorted version of X generated by a privacy preserving mechanism. The set could then represent 
a set of functions that are known to be easy to infer from Y dne to inherent utility constraints of 
the setup. In particular, as will be shown in Section [6.41 we will consider the fnnctions in <1> as the 
individual symbols of the plaintext. In this case, the resnlts introduced in this section are nsed to 
derive bonnds on the MMSE of reconstrncting a target fnnction of the plaintext in terms of the 
symbol-secrecy achieved by the nnderlying list-source code given by the encryption scheme. This 
result extends symbol secrecy to a broader setting. 


6.1 Lower Bounds for MMSE 

The resnlts introdnced in this section are based on the following Lemma. 
Lemma 3. Let Zn : (0, oo)"' x [0,1]"' —)• M 6 e given by 

Zn{ai, b) = max {a^y |y G \\y\\2< 1, y < b} . 


( 11 ) 


Let IT be a permutation of (1,2,..., n) such that 6 ^(i)/a 7 r(i) < ... < biT{n)/<^iT{n) ■ U ^ 7 r(i)/® 7 r(i) > 
1, z„(a, b) = ||a|| 2 . Otherwise, 


i(a, b) ^ ^ ^7 


(d 


2=1 


-f 




|a||2-\ a2 


2=1 


2=1 


(d 


where 


= max < 

k G [n] 

^7T{k) ^ 

(i-EtdL,)* 

1/ 



< 




Proof. The proof is given in the appendix. 


( 12 ) 


(13) 


□ 


Throughout this section we assume <1 C C 2 {px) and E [(fi{X)(j)j{X)] = 0 for i / j. Furthermore, 
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let V be an observed variable that is dependent of X, and for a given the inequality 


max E [0,(X)V>(y)] = ||E [4>i{X)\Y] || 2 < A, 
'4’€C2(py) 


is satished, where 0 < Aj < 1. This is equivalent to mmse(0j(X)|y) > 1 — A^. 

Theorem 4. Let |E [(/)(X)(^i(X)] |= pi > 0. Denoting p = {\pi\,... ,\pm\), A = (Ai,...,Am), 


Po = - Yd=i pf, Ao = 1 Po - (Po, p) and Ao = (Ao, A), then 


||E[</.(y)|y]||2<%|(Po,Ao), 

(14) 

where 


D / \ ^A 1 (Po’-^o), «/Po > 0, 

^|$|(po>^o) = < 

l 2 :|$|(p,A), otherwise. 

(15) 

and Zn is given in (flTl). Consequently, 


mmse((?i(X)|y) > 1 - S|,j,|(po, Ao)^. 

(16) 


Proof. Let h{X) = pQ^{(j){X) — if Po > 0, otherwise h{X) = 0. Note that h{X) € 

C 2 {px)- Then for •0 G P^ 2 {py) 


\K[^{X)^|;{Y)]\ 


poE [h{X)^fiY)] + ^ piE [UX)m)] 

i=l 


< PO |E [h{X)f;{Y)]\ + Y,\Pr^ [UXmY)]\ 

i=l 


= PO |E UX)iTxi’){X)] I + ^ |p,E [UX)iTxi’){X)] \ . 

i=l 


Denoting \E[h{X){Tx'ip){X)]\= xq, \E[(j)i{X){Tx'if){X)]\= Xi, x = {xo,xi,... ,Xm), and p = 
(po, |pi|,..., |pml), the last inequality can be rewritten as 


|E[^(x)V^(y)]|<p^x. 


(17) 


Observe that ||x|| 2 < 1 and x* < Aj for i = 0 ,..., m, and the right hand side of (fT7|) can be 
maximized over all values of x that satisfy these constraints. We assume, without loss of generality, 
that Po > 0 (otherwise set xq = 0 ). The left-hand side of (fT71) can be further bounded by 


|E[(/)(y)v^(y)]| <z™+i(Po,Ao), ( 18 ) 

where A = (1, Ai,..., Am) and Zm+i is dehned in (fTT]l . The result follows directly from Lemma [3] 
and noting that max. 0 gp 2 (py) E [</’(^)V’(^)] = l|E [(?i(X)|y] || 2 . □ 
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Denote V’i — Ty 4>i / \\Ty (i)i \\2 and (poiX) = {(j){X) — Pi4>i{X)) / ■ The previous bound 

can be further improved when E [tpi{Y)(pj{X)] = 0 for i 7 ^ j, j G {0,, m}. 

Theorem 5. Let |E [(/)(X)(/)j(X)] |= pj > 0 for cpi G <h. In addition, assume E ['(/’i(T)^j(y)] = 0 
for i ^ j, i G [t] and j € {0,... , |d>|}, where 0 < t < |d>|. Then 


mmx)\Y] ||2< 


\ 


xf Pi + 


/ j % rt 

k=l 


(19) 


where p = {po, pt,..., Pm), A = (1, At,, A^) and Bm is defined in (fT^ (considering Bq = Q). In 
particular, if t = m. 


\^\ 

\mfix)\Y]h<^ pi + Y^xhl 




( 20 ) 


k=l 


and this bound is tight when po = 0. Furthermore, 

t _ 2 

^.{fiX)\Y)>l-Y,Xfpf-B^^^_i{p,x) . 


mmse 


( 21 ) 


k=l 


Proof. For any ip G T 2 (py), let a* = E [ip(Y)ipi(Y)] and ipo(Y) = {i’iY)-Yf\=i aiipi(Y))aQ \ where 
“0 = (1 - Observe that ipo G C 2 {py) and E [(pi{X)ipj{Y)] = E [ipi{Y)ipj{Y)] = 0 for 

* / J, i G {0,... , |d>|} and j G [t]. Consequently 


E[cP{X)iP{Y)]=E 


1-3^1 \ / t 

'^Pi<Pi{X)\ J^ajV’i(y) 
i=o j \j=0 

I'l’l t 

EE PiajE[UX)iPj(Y)] 

i=0 j=0 

m 

ao Y1 Pi^[MX)My)] 

i=0,i^[n] 

^ t 

< |ao|-B|$|_t (p,X^ + '^\XiPiai 
i=l 


< 


+ Ei^ ipiOti I 


i=l 


< 


\ (P’y ■ 

\l i=l 


( 22 ) 

(23) 


Inequality (f 2 ^ follows from the bound (fTT)) . and (f23ll follows by observing that X)i=o^f ~ ^ 
applying the Cauchy-Schwarz inequality. 


Finally, when po = 0, ([251) can be achieved with equality by taking ip = Yfi 

V Ei KPi 


□ 
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The following three, diverse examples illustrate different usage cases of Theorems U] and [5j 
Example [5] illustrates Theorem [5] for the binary symmetric channel. In this case, the basis can 
be conveniently expressed as the parity bits of the input to the channel. Example [ 6 ] illustrates how 
Theorem 0 can be applied to the g-ary symmetric channel, and demonstrates that bound (|2Up is 
sharp. Finally, Example [7] then illustrates Theorem 0] for the specific case where all the values pi 
and Aj are equal. 

Example 5 (Binary Symmetric Channel). Let X = { — 1,1} and y = {—1,1}, and be the result 
of passing through a memoryless binary symmetric channel with crossover probability e. We 
also assume that is composed by n uniform and i.i.d. bits. For S C [n], let xs{^"') — 

Any function (/) : A —)• M can then be decomposed in terms of the basis of functions as |44] 

0(X-) = ^ csxsixn, 

5C[n] 

where cs = 'E[(l){X'^)xs{X'^)]. Furthermore, since E [x 5 (Ai"')|y"'] = (1 — 2e)l‘^l, it follows from 
Theorem [5] that 

mmse(,/.(X’")|y") = 1-^1 ^si'^ “ 26)21*^1. (24) 

<SC[n] 

This result can be generalized for the case where X^ = Y^ ® Z"', where the operation ® denotes 
bit-wise multiplication, is drawn from {—1,1}”" and X"' is uniformly distributed. In this case 

mmse(,^(A-)|y-) = I - ^ c|E [xsiZ^f ■ (25) 

<SC[n] 

This example will be revisited in Section 16.31 where we restrict to be a binary function. 

Example 6 (( 7 -ary symmetric channel). For A = T = [( 7 ], an (e, g)-ary symmetric channel is defined 
by the transition probability 

PY\xiy\x) = (1 - €)ly=x +e/q. (26) 

Any function G C 2 {px) such that E [( 7 !)j(X)] = 0 satisfies 

UY) = TyHX) = (1 - e)(/.(y). 


and, consequently, ||Ty( 7 !)(X)|| 2 = (1 —e). We shall use this fact to show that the bound (f20]l is sharp 
in this case. 

Observe that for 4>i,(l)j G C 2 {px), if ^[4>iiX)4>jiX)] = 0 then M['ijji{Y)'il>j(Y)] = 0. Now let 
4> G C 2 {px) satisfy E [(/)(Ar)] = 0 and E [4>{X)4>i{X)] = pi for 4>i G 4*, where |4>|= m, satisfies the 
conditions in Theorem^ and YliPi — f- addition, ||'!/’i|| 2 = (1 ~ e) = Then, from (|20|) . 


\\TY(^{X)h< 
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= 1 - e, 

which matches ||ry(/>(X)|| 2 , and the bound is tight in this case. 

Example 7 (Equal MMSE and correlation). We now turn our attention to Theorem 01 Consider 
the case when the correlations of (p with the references functions (pi are all the same, and each (pi 
can be estimated with the same MMSE, i.e. Ai = • • • = Am = A and pf = ■ ■ ■ = , p > Q and 

A^ < < 1 /m. Then bound (1141) becomes 

||E [(p{X)\Y] || 2 < mXp + v^(l -mp2)(l -mA2). 


6.2 One-Bit Functions 


Let X be a hidden random variable and T be a noisy observation of X. Here we denote = {(pi}^i 
a collection of m predicates of X, where T) = <pi{X), (pi : X {—1,1} for i G [m] and, without 
loss of generality E [T)] = 6* > 0. 

We denote by T) an estimate of T) given an observation of Y, where Fi ^ X ^ Y Fi- We 
assume that for any Fi 


nFiPi] 


<l-2ai 


for some 0 < a* < (1 —6j)/2 < 1/2. This condition is equivalent to imposing that PrjFj p Fi} > ai, 
since 


E 



Pr{F, = Fi} - Pr{F, / FJ 


= l-2Pr{F ^Fj. 


In particular, this captures how well F can be guessed based solely on an observation of Y. 

Now assume there is a bit F = (p(Y) such that E [FFj] = pi for i G [m] and E [FjFj] = 0 for 
i p j. We can apply the same method used in the proof of Theorem |4] to bound the probability of 
F being guessed correctly from an observation of Y. 

Corollary 1. For A* = 1 — 2ai, 

Pr(F/F)>i(l-F|,i,|(p,A)). (27) 

Proof. The proof follows the same steps as Theorem 01 (p{Y) G C2{py)- Cl 

In the case m = 1, we obtain the following simpler bound, presented in Proposition [2l which 
depends on the following Lemma. 


22 








Lemma 4. For any random variables A, B and C 


Pr(A ^B)< Pr(^ ^C)+ Vi{B / C). 


Proof. 


Pr(^ ^B)= Pi{A ^BAB = C)+ Pr(A ^ B A B ^ C) 

= Pt{A ^CAB = C) + Pt{B / C) Pr(^ / B\B / C) 
< Pv{A / C) + Pt{B + C). 


□ 


Proposition 2. IfPi{Fi ^ Fi) > a for all Fi and E [-FPi] = p > 0. Then for any estimator F 


Pr(F / F) > 



Proof. From Lemma HI 


Pr(F / F) > (Pt{Fi ^ F)- Pr(Fi / F 

1- p 


> 


— a 


(28) 


□ 


6.3 One-Time Pad Encryption of Functions with Boolean Inputs 

We return to the setting where a legitimate transmitter (Alice) wishes to communicate a plaintext 
message to a legitimate receiver (Bob) through a channel observed by an eavesdropper (Eve). 
Both Alice and Bob share a secret key K that is not known by Eve. Alice and Bob use a symmetric 
key encryption scheme determined by the pair of encryption and decryption functions (Enc, Dec), 
where Y'^ = Enc(A"',Fr) and X'^ = Dec(y"',iP). Here we assume that both the ciphertext and the 
plaintext have the same length. 

We use the results derived in the previous section to assess the security properties of the one¬ 
time pad with non-uniform key distribution when no assumptions are made on the computational 
resources available to Eve. In this case, perfect secrecy (i.e. I{X'^;Y"') = 0) can only be achieved 
when H{K) > H{X'^) [3], which, in turn, is challenging in practice. Nevertheless, as we shall show 
in this section, information-theoretic security claims can still be made in the short key regime, i.e. 
H{K) < F[{X^). We first prove the following ancillary result. 

Lemma 5. Let F be a Boolean random variable and F ^ X ^ Y -A F, where |T|> 2. Further¬ 
more, Pr{F / F} > a for all Y ^ F. Then I{F;Y) < 1 — 2a. 
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Proof. The result is a direct consequence of the fact that the channel with binary input and finite 
output alphabet that maximizes mutual information for a fixed error probability is the erasure 
channel, proved next. Assume, without loss of generality, that y = [m] and (—1, y) > u) 

for y G [k] and pFy{—l,y) < pF,y(l,y) for y G {k + where k G [m]. Now let T be a 

random variable that takes values in [2m] such that 


PFy{b,y) 


PF,Y{h,y) -pF,y(l,y) 
PF,Y{h,y) -pF,y(-l,y) 

PF,y(l,y) 

PF,y(-l,y) 

\ 


y G [^ 1 , 

y G {/c + 1,... ,m}, 
y — m G [k], 
y — m G {k + 1,..., m} 


Note that F ^ Y ^ Y, since Y = Y — and, consequently, I{F;Y) > I{F;Y). 

Furthermore, the reader can verify that 

min Pr{F / F} = min Pr{F ^ F} = a. 

Y^F Y^F 

In particular, given the optimal estimator T —)■ F, a detection error can only occur when Y G 
{k + 1,, m}, in which case F = F with probability 1/2. 

Finally, 


H{F\Y) = - Py(y)PF|y(^|y)logPF|y(^l2f) 

ye[2m] 

= Y Pviy) 

y€:{m+l,2m} 

> 2a. 

Consequently, I{F;Y) = H{F) — H{F\Y) < 1 — 2a. The result follows. □ 

Let A” be a plaintext message composed by a sequence of n bits drawn from {—1,1}"". The 
plaintext can be perfectly hidden by using a one-time pad; A ciphertext Y'^ is produced as Y'^ = 
A” ® .Z’”, where the key K = Z” is a uniformly distributed sequence of n i.i.d. bits chosen 
independently from A"^. The one-time pad is impractical since, as mentioned, it requires Alice and 
Bob to share a very long key. 

Instead of trying to hide the entire plaintext message, assume that Alice and Bob wish to hide 
only a set of functions of the plaintext from Eve. In particular, we denote this set of functions as 
$ = {</i,..., where (fi : {-1,1}- ^ {-1,1}, E [</i(A-)] = 0 and E [,/i(A-),/,(A-)] = 0. The 
set of functions $ is said to be hidden /((/)j(A"'); A"') = 0 for all (fi G <h. Can this be accomplished 
with a key that satishes H{K) <C F(A")? 
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The answer is positive, but it depends on <h. We denote the Fourier expansion of G as 


^ Pi,SXS- 
<SC[n] 

The following result shows that (pi is perfectly hidden from Eve if and only if = 0 

for all xs such that pi^s > 0- 

Lemma 6. If I{(j)i{X^)-,Y^) = 0 for all (pi G then /(x 5 (X"'); T"-) = 0 for all S such that 
Pi,S > 0 for some i G [m]. 

Proof. Assume that /(x 5 (A"^); y"") > 0 for a given pi^g > 0. Then there exists b : ^ {—1,1} 

such that E [6(y"')x<s(y"')] = A > 0. Consequently, from ([20]) . E[b{Yy(pi{X'^)] > Xpi^s > 0, and 
(pi{X'^) is not independent of y”. □ 

The previous result shows that hiding a set of functions perfectly, or even a single function, 
might be as hard as hiding A”. Indeed, if there is a G such that E [(pi^XyXsiXy] > 0 for 
all S C [n] where |5|= 1, then perfectly hiding this set of functions can only be accomplished by 
using a one-time pad. Nevertheless, if we step back from perfect secrecy, a large class of functions 
can be hidden with a comparably small key, as in the next example. 

Example 8 (BSC revisited). Let Z” be a sequence of n i.i.d. bits such that Pr{Zj = —1} = e, and 
consider once again the one-time pad Y"^ = A” ® . Furthermore, denote 

ckfc = {0 : {-1, ir ^ {-1,1} I E \f>{xyxs{xy\ = 0 V|5|< k} . 

Let 0 G and 0(A”) = X] 5 .| 5 |>fc p<sX5(-^"')- Then, from Theorem [5] and Corollary [H for any 
Pr{^.(X”) 6(r‘)} >1^1- / E P|(l - 2£)2I«I j 

Consequently, from LemmaEl I{(p{X^)-,Y^) < (1 — 2e)^ for all (p G ^k- Note that H{Zy = nh{e), 
which can be made very small compared to n. Therefore, even with a small key, a large class of 
functions can be almost perfectly hidden from the eavesdropper through this simple one-time pad 
scheme. The BSC setting discussed in Example [5] is generalized in the following theorem which, in 
turn, is a particular case of the analysis in [45] . 

Theorem 6 (Generalized One-time Pad). Let Y^ = X^ 0 Z'^, A"" X Z'^, A” be uniformly 
distributed, (p : {-1,1}” ^ {-1,1} and 4>{Xy = Y^sQn]PsXs{Xy. We define cg = E[x5(^”')] 
for 5 C [n]. Then 

I{i,(X’‘y,Y’')< Xi^spsP. ( 29 ) 

V 
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In particular, I{(p{X^);Y"^) = 0 if and only if cs = 0 for all S such that ps / 0. 

Proof. Let : {-1,1}” -5- {-1,1} and ii{Y'^) = E^cfn] ^<5X5(1^”)• Note that E^cfn]'^1 = 1- 
Then 


E [,/.(x”)v^(y”)] = E [(/.(x”)E [v^(y”)|x”]] 

= E cfix^) Y, dsnxs{Y^)m 

5C[n] 


= E 


^ dsnxsix^ ® Z^)\X^] 

<SC[n] 


= E 


dsnxs{xnxs{z^)\x^ 

5C[n] 

dsn^{x^)xs{x'^)]nxs{z'^)] 

5C[n] 

Y ^SPSCS 

5C[n] 


V 


(30) 

(31) 


where m follows from the Cauchy-Schwarz inequality. The inequality (I29p then follows from 
Lemma O Finally, assume there exists S C [n] such that both cs ^ 0 and ps 0. Then 
setting if{Y^) = xs(X^)i it follows from ((31)11 that E [())(X”)V'(y”)] = pscs / 0 and, consequently, 

/(0(X”);y”) > 0. □ 


6.4 From Symbol Secrecy to Function Secrecy 

Symbol secrecy captures the amount of information that an encryption scheme leaks about in¬ 
dividual symbols of a message. A given encryption scheme can achieve a high level of (weak) 
information-theoretic security, but low symbol secrecy. As illustrated in Section [4.11 by sending a 
constant fraction of the message in the clear, the average amount of information about the plain¬ 
text that leaks relative to the length of the message can be made arbitrarily small, nevertheless the 
symbol secrecy performance is always constant (i.e. does not decrease with message length). 

When X is uniformly drawn from Eg for which an [n,k,n — k + 1) MDS code exists, then an 
absolute symbol secrecy oikjn can always be achieved using the encryption scheme suggested in 
Proposition [TJ If A is a binary random variable, then we can map sequences of plaintext bits of 
length [log 2 q\ to an appropriate symbol in Eg, and then use the parity check matrix of an MDS 
code to achieve a high symbol secrecy. Therefore, we may assume without loss of generality that 
A” is drawn from {—1,1}”. We also make the assumption that A” is uniformly distributed. This 
can be regarded as an approximation for the distribution of A” when it is, for example, the output 
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of an optimal source encoder with sufficiently large blocklength. 


Theorem 7. Let be a uniformly distributed sequence of n hits, Y = Enc„(X”, ii'), and Ue and 
the corresponding symbol secrecy and dual symbol secrecy o/EnCn, defined in © and @ , respec¬ 
tively. Furthermore, for 4> : { — 1,1}"' —>• {—1,1} and E [(/>(X")] = 0, /et(/>(X") = Ylsc[n]PsXsi^"')- 
Then for any 4> : y ^ {“I; 1} 

Pr{(/)(X") ^{Y)} > ^ (1 - d?|$|(p, A)) , (32) 

where <^ = {xs ■ ps ^ 0}, X{t) = /i^^((l - elt)+), A = {A(|5|)}5c[n] and p = {|p 5 |} 5 c[n]- In 
particular, 

Pr{^,(X”) / ^(r)} > 1 ( 1 - / Pi ] . (33) 

\ Y l‘5|>nMo / 

Proof. From the definition of symbol secrecy, for any S C [n] with |5|= t 

I{xs{X^);Y)<IiX^-Y)<e:t, 


and, consequently. 


H{xs{X^)\Y)>{l-e:t)+. 

From Fano’s inequality, for any binary F where Y ^ F 

PT{xs{Xn^F}>h^\{l-e:t)+), 

where h^^ : [0,1] [0,1/2] is the inverse of the binary entropy function. In particular, from the 

definition of absolute symbol secrecy, if e/ = 0, then 

Pv{xs{Xn7lF} = l/2 V|5|<n/io. 

The result then follows directly from Theorem [5l the fact that 4>{X^) = Ylsc[n] PsXsiX^) and 
letting X{t) = n 

7 Discussion 

In this section we discuss the application of our results to different settings in privacy and cryptog¬ 
raphy. 

7.1 The Correlation-Error Product 

We momentarily diverge from the cryptographic setting and introduce the error-correlation product 
for the privacy setting considered by Calmon and Fawaz in m- Let W and X be two random 
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variables with joint distribution pw,x- hh represents a variable that is supposed to remain private, 
while X represents a variable that will be released to an untrusted data collector in order to receive 
some utility based on X. The goal is to design a randomized mapping Py\Xj called the privacy 
assuring mapping, that transforms X into an output Y that will be disclosed to a third party. 

The goal of a privacy assuring mechanism is to produce an output Y, derived from X according 
to the mapping Py\x^ be released to the data collector in the place of X. The released 

variable Y is chosen such that W cannot be inferred reliably given an observation of Y. Simultane¬ 
ously, given an appropriate distortion metric, X should be close enough to Y so that a certain level 
of utility can still be provided. For example, W could be a user’s political preference, and X a set of 
movie ratings released to a recommender system in order to receive movie recommendations. Y is 
chosen as a perturbed version of the movie recommendations so that the user’s political preference 
is obscured, while meaningful recommendations can still be provided. 

Given W ^ X ^Y and pw,x , a privacy assuring mapping is given by the conditional distribu¬ 
tion Py\x- The choice of Py\x determines the tradeoff between privacy and utility. If Py\x = Py, 
then perfect privacy is achieved (i.e. W and Y are independent), but no utility can be provided. 
Conversely, if Py\x is the identity mapping, then no privacy is gained, but the highest level of 
utility can be provided. 

When W = 4>{X) where 4> G C 2 {px)-, the bounds from Section [6T] shed light on the fundamental 
privacy-utility tradeoff. Returning to the notation of Section [6Tl let W = (t){X) be correlated with 
a set of functions The next result is a direct corollary of Theorem [5j 

Corollary 2. Tef E [IT(/)i(X)] = pi, YjfXPi = V’i(T) = E [<j)i{X)\Y] and, fori ^ j, E [(j)i{X)(t)j{X)] 
0 and E ['0i(y)V’j(T)] = 0. Then 

m 

mmse(IT|y) = ^ mmse((/>j(y)|X)/??. (34) 

i=l 

We call the product mmse((/>j(y)|y)/9? the error-correlation product. The secret variable W 
cannot be estimated with low MMSE from Y if and only if the functions that are strongly 
correlated with W (i.e. large pf) cannot be estimated reliably. Consequently, if pi is large and 
(pi is relevant for the utility provided by the data collector, privacy cannot be achieved without a 
significant loss of utility: mmse(^j(X)|y) is necessarily large if mmse(iy|y) is large. Conversely, in 
order to hide VF, it is sufficient to hide the functions pi{X) that are strongly correlated with <p{X). 
This no-free-lunch result is intuitive, since one would expect that privacy cannot be achieved if 
utility is based on data that is strongly correlated with the private variables. The results presented 
here prove that this is indeed the case. 

We present next a general description of a two-phase secure communication scheme for the 
threat model described in Section 11.51 presented in terms of the list-source code constructions 
derived using linear codes. Note that this scheme can be easily extended to any list-source code 
by using the corresponding encoding/decoding functions instead of multiplication by parity check 
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matrices. 


7.2 A Secure Communication Scheme Based on List-Source Codes 

We assume that Alice and Bob have access to a symmetric-key encryption/decryption scheme 
(Enc^, Dec^) that is used with the shared secret key K and is sufficiently secure against the adver¬ 
sary. This scheme can be, for example, a one-time pad. The encryption/decryption procedure is 
performed as follows, and will be used as components of the overall encryption scheme (Enc, Dec) 
described below. 

Scheme 2. Input: The source encoded sequence x E F” , parity check matrix H of a linear code in 
Fq, a full-rank k x n matrix D such that rank([H^ D^]) = n, and encryption/decryption functions 
(Enc^, Dec^). We assume both Alice and Bob share a secret key K. 

Encryption (Enc): 

Phase I (pre-caching): Alice generates cr = Hx and sends to BobJ§ 

Phase II (send encrypted data): Alice generates e = Enc'(Dx, A) and sends to Bob. 

Decryption (Dec): Bob calculates Dec^(e, AT) = Dx and recovers x from a and Dx. 

Assuming that (Enc', Dec') is secure, the information-theoretic security of Scheme [2] reduces 
to the security of the underlying list-source code (i.e. Scheme d]). In practice, the encryp¬ 
tion/decryption functions (Enc', Dec') may depend on a secret or public/private key, as long as 
it provide sufficient security for the desired application. In addition, assuming that the source 
sequence is uniform and i.i.d. in F^, we can use MDS codes to make strong security guarantees, 
as described in the next section. In this case, an adversary that observes cr cannot infer any 
information about any set of k symbols of the original message. 

Note that this scheme has a tunable level of secrecy: The amount of data sent in phase I and 
phase II can be appropriately selected to match the properties of the encryption scheme available, 
the size of the key length, and the desired level of secrecy. Furthermore, when the encryption 
procedure has a higher computational cost than the list-source encoding/decoding operations, list- 
source codes can be used to reduce the total number of operations required by allowing encryption 
of a smaller portion of the message (phase II). 

The protocol outline presented in Scheme [2] is useful in different practical scenarios, which are 
discussed in the following sections. Most of the advantages of the suggested scheme stem from 
the fact that list-source codes are key-independent, allowing content to be distributed when a key 
distribution infrastructure is not yet established, and providing an additional level of security if 
keys are compromised before phase II in Scheme [21 

^Here, Alice can use message authentication codes and public key encryption to augment security. Furthermore, 
the list-source coding scheme can be used as an additional layer of security with information-theoretic guarantees in 
symmetric-key ciphers. Since we are interested in the information-theoretic security properties of the scheme, we will 
not go into further details. We do recognize that in order to use this scheme in practice additional steps are needed 
to meet modern cryptographic standards. 
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7.3 Content pre-caching 

As hinted earlier, list-source codes provide a secure mechanism for content pre-caching when a key 
infrastructure has not yet been established. A large fraction of the data can be list-source coded and 
securely transmitted before the termination of the key distribution protocol. This is particularly 
significant in large networks with hundreds of mobile nodes, where key management protocols can 
require a significant amount of time to complete |46] . Scheme [2] circumvents the communication 
delays incurred by key compromise detection, revocation and redistribution by allowing data to be 
efficiently distributed concurrently with the key distribution protocol, while maintaining a level of 
security determined by the underlying list-source code. 

7.4 Application to key distribution protocols 

List-source codes can also provide additional robustness to key compromise. If the secret key is 
compromised before phase II of Schemed the data will still be as secure as the underlying list-source 
code. Even if a (computationally unbounded) adversary has perfect knowledge of the key, until 
the last part of the data is transmitted the best he can do is reduce the number of possible inputs 
to an exponentially large list. In contrast, if a stream cipher based on a pseudo-random number 
generator were used and the initial seed was leaked to an adversary, all the data transmitted up 
to the point where the compromise was detected would be vulnerable. The use of list-source codes 
provide an additional, information-theoretic level of security to the data up to the point where the 
last fraction of the message is transmitted. This also allows decisions as to which receivers will be 
allowed to decrypt the data can be delayed until the very end of the transmission, providing more 
time for detection of unauthorized receivers and allowing a larger flexibility in key distribution. 

In addition, if the level of security provided by the list-source code is considered sufficient and the 
key is compromised before phase II, the key can be redistributed without the need of retransmitting 
the entire data. As soon as the keys are reestablished, the transmitter simply encrypts the remaining 
part of the data in phase II with the new key. 

7.5 Additional layer of security 

We also highlight that list-source codes can be used to provide an additional layer of security 
to the underlying encryption scheme. The message can be list-source coded after encryption and 
transmitted in two phases, as in Scheme[2l As argued in the previous point, this provides additional 
robustness against key compromise, in particular when a compromised key can reveal a large amount 
of information about an incomplete message (e.g. stream ciphers). Consequently, list-source codes 
are a simple, practical way of augmenting the security of current encryption schemes. 

One example application is to combine list-source codes with stream ciphers. The source-coded 
message can be initially encrypted using a pseudorandom number generator (PRG) initialized with 
a randomly selected seed, and then list-source coded. The initial random seed would be part of the 
encrypted message sent in the hnal transmission phase. This setup has the advantage of augmenting 
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the security of the underlying stream cipher, and provides randomization to the list-source coded 
message. In particular, if the LSC is based on MDS codes and assuming that the distribution of the 
plaintext is nearly uniform, strong information-theoretic symbol secrecy guarantees can be made 
about the transmitted data, as discussed in Section [5J Even if the underlying PRG is compromised, 
the message would still be secure. 

7.6 Tunable level of secrecy 

List-source codes provide a tunable level of secrecy, i.e. the amount of security provided by the 
scheme can be adjusted according to the application of interest. This can be done by appropriately 
selecting the size of the list (L) of the underlying code, which determines the amount of uncertainty 
an adversary will have regarding the input message. In the proposed implementation using linear 
codes, this corresponds to choosing the size of the parity check matrix H, or, analogously, the 
parameters of the underlying error-correcting code. In terms of Scheme [21 a larger (respectively 
smaller) value of L will lead to a smaller (larger) list-source coded message in phase I and a larger 
(smaller) encryption burden in phase II. 

8 Conclusions 

We conclude the paper with a summary of our contributions. We introduce the concept of LSCs, 
which are codes that compress a source below its entropy rate. We derived fundamental bounds for 
the rate list region, and provided code constructions that achieve these bounds. List-source codes 
are a useful tool for understanding how to perform encryption when the (random) key length is 
smaller than the message entropy. When the key is small, we can reduce an adversary’s uncertainty 
to a near-uniformly distributed list of possible source sequences with an exponential (in terms of the 
key length) number of elements by using list-source codes. We also demonstrated how list-source 
codes can be implemented using standard linear codes. 

Furthermore, we presented a new information-theoretic metric of secrecy, namely e-symbol 
secrecy, which characterizes the amount of information leaked about specihc symbols of the source 
given an encoded version of the message. We derived fundamental bounds for e-symbol secrecy, 
and showed how these bounds can be achieved using MDS codes when the source is uniformly 
distributed. 

We also introduced results for bounding the probability that an adversary correctly guesses a 
predicate of the plaintext in terms of the symbol secrecy achieved by the underlying encryption 
scheme. These results are based on Lemma [Sj which, in turn, was used to derive bounds on 
the information leakage of a security system that does not achieve perfect secrecy. These bounds 
provide insight on how to design symmetric-key encryption schemes that hide specific functions of 
the data, where uncertainty is captured in terms of minimum-mean squared error. These results 
also shed light on the fundamental privacy-utility tradeoff in privacy systems. 
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Appendix A Proof of Lemma [3] 

For fixed a, b E M" where Oj > 0 and 6* > 0, let zp : — )• M and zp : 

zp{y) = a^y, 

Z£)(u) = a’^b u^b ||u||2. 


be given by 


Furthermore, we define M(a) = {u E > a} and i3(b) = {y E M” | ||y|| 2 < IjY < b}. 

The optimal value 2 ;„(a, b) is given by the following pair of primal-dual convex programs: 

Zn(&,h) = max zp(y) = min zn(u). 

yeB(b) u&Aia) 

Assume, without loss of generality, that 6 i/ai < 62/02 <••• < bnlun, and let k* be defined in (fT3]l . 
Let Cj = \ \ ■ Note that since bf < 1, we have c^* > 0. In addition, let 


y — ( 61 ,..., 6 fc», ,..., a-nCk *) 


and 


U — ( 61 /Cfc* , . . . , bjf,* jCk* , Clfc*-|-i, . . . , On) ■ 

From the definition of k*, y* E ^(b) and u* E M(a). Furthermore, 


T, * 


zpiy*) = a y 


'^aibiP ^ Ck*aj 
i=l i=k*+l 


aih + 


i=l 




|a ||2 




2=1 


2=1 
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and 


zd{u*) =a'^b -|- u*'' b -|- ||u* 
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=E 





+ c 


-1 

fc* ^ 


k* 

E 

i=l 


b‘j + c 



k* 

= Oi&i + 
i=l 

k* 

= ^ ttibi + 
i=l 

=zp{y*). 



Since both the primal and the dual achieve the same value at y* and u*, respectively, it follows 
that the value zp{y*) given in ([55]) is optimal. 
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